On narrowing the gap between verification and systematic testing

Our work on narrowing the gap between verification and systematic testing has two directions: (1) complementing verification with systematic testing, and (2) pushing systematic testing toward reaching verification. In the first direction, we explore how to effectively combine static analysis with systematic testing, so as to guide test generation toward properties that have not been previously checked by a static analyzer in a sound way. This combination significantly reduces the test effort while checking more unverified properties. In the second direction, we push systematic testing toward checking as many executions as possible of a real and complex image parser, so as to prove the absence of a certain class of errors. This verification attempt required no static analysis or source code annotations; our purely dynamic techniques targeted the verification of the parser implementation, including complicated assembly patterns that most static analyses cannot handle. ACM CCS: Software and its engineering → Software creation and management → Software verification and validation → Formal software verification Software and its engineering → Software creation and management → Software verification and validation → Software defect analysis → Software testing and debugging